Dwelling on the Negative: Incentivizing Effort in Peer Prediction
نویسندگان
چکیده
Agents are asked to rank two objects in a setting where effort is costly and agents differ in quality (which is the probability that they can identify the correct, ground truth, ranking). We study simple output-agreement mechanisms that pay an agent in the case she agrees with the report of another, and potentially penalizes for disagreement through a negative payment. Assuming access to a quality oracle, able to determine whether an agent’s quality is above a given threshold, we design a payment scheme that aligns incentives so that agents whose quality is above this threshold participate and invest effort. Precluding negative payments leads the expected cost of this quality-oracle mechanism to increase by a factor of 2 to 5 relative to allowing both positive and negative payments. Dropping the assumption about access to a quality oracle, we further show that negative payments can be used to make agents with quality lower than the quality threshold choose to not to participate, while those above continue to participate and invest effort. Through the appropriate choice of payments, any design threshold can be achieved. This selfselection mechanism has the same expected cost as the costminimal quality-oracle mechanism, and thus when using the self-selection mechanism, perfect screening comes for free.
منابع مشابه
Incentivizing Peer Review
Since the 1700s, peer review has been a part of scientific publication. Quality peer reviews take extensive time and effort yet reviewers are not often compensated. This editorial announces the new offering of the National Certification Board for Thereputic Massage & Bodywork (NCBTMB)-approved continuing education hours for our peer reviewers who complete timely reviews. Instructions to registe...
متن کاملPrediction of Substance Use Tendency based on Outcome Expectancies, Parental Monitoring, and Resistance to Peer Influences
Objective: The present research aimed to predict substance use tendency based on outcome expectancies, parental monitoring, and resistance to peer influences. Method: The research method was descriptive-correlational. The statistical population of the research included all students of Shahid Beheshti University of Tehran in 2019-2020, of which 419 people were selected using convenience sampling...
متن کاملبررسی رابطه کمال گرایی بهنجار و نوروتیک با خودناتوان سازی و زیر مقیاس های آن در دانشجویان پسر
Aim: The present research was aimed to determine the relationship between normal and neurotic perfectionism, and self-handicapping and its subscales (negative mood, effort, and excuse-making) among male university students. Method: The method of the current research was descriptive-correlational. Total of 350 male post-graduate students from University of Tehran were selected by cluster samplin...
متن کاملPeer Reviewers’ Comments on Research Articles Submitted by Iranian Researchers
The invisible hands of peer reviewers play a determining role in the eventual fate of submissions to international English-medium journals. This study builds on the assumption that non-native researchers and prospective academic authors may find the whole strive for publication, and more specifically, the tough review process, less threatening if they are aware of journal reviewers’ expectation...
متن کاملIncentivizing Evaluation via Limited Access to Ground Truth: Peer-Prediction Makes Things Worse
In many settings, an effective way of evaluating objects of interest is to collect evaluations from dispersed individuals and to aggregate these evaluations together. Some examples are categorizing online content and evaluating student assignments via peer grading. For this data science problem, one challenge is to motivate participants to conduct such evaluations carefully and to report them h...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2013